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(54) A method for producing an image to be transmitted from a terminal and the terminal 



(57) In the invention, devices and a method are de- this is located, a sub-section (62) of the section the cam- 
scribed by means of wh ich it is possible to focus a cam- era ( 11 ) has stored in an image memory (64) is selected, 
era (11) placed in a terminal (10), particularly in an im- the sub-section containing essentially the upper body of 
age phone using radio communication, at a user of the the user of the terminal. Alternatively, the camera (11) 
terminal (10). The focusing of the camera (11 ) Is imple- is focused, e.g., by means of electric motors, at the user 
mented, e.g., by means of pattem recognition (61), or the user himself is asked to turn the terminal (10) in 
whereupon the terminal (10), according to the invention, the required direction, e.g., with the help of indication 
searches for the figure of an upper body (1 8) of the user characters transmitted on to the display of the terminal 
of the terminal in the image field recorded by it. When (10). 
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Description 

OBJECT OF THE INVENTION 

5 The present rnventbn relates to a method and a temninal tor producing an innage. to be transnnitted, in the terminal 

wherein a picture of a user of the terminal is recorded and an image, intended for further transmission, is produced 
from the recorded image field and further transmitted to a receiver. The invention relates particularly to the focusing of 
a video camera or a still camera on the user of a portable image phone using radio communication. 

10 BACKGROUND OF THE INVENTION 

The next major step for wireless communication will probably be the transfer of a moving video image. However, 
due to the insufficient transmission capacity of wireless telecommunication networks and the undeveloped state of 
video image coding technique, present technology does not yet enable to use of these kinds of portable terminals. 

15 When studying the requirements set on a terminal capable of processing a vkieo image, certain new problems become 
apparent, A problematic situatkjn occurs when using a portable vkleophone that shoots the face of a user from the* 
front while the user is hoWing the device in his hand. In this case, the problem is the focusing of the camera, located 
within the image phone, on the object to be recorded, typkially, on the user's face. When the user is holding the image 
phone in his hand, it is very likety that, due to the movements of his hand and head, it ^ very difficult to record the 

20 object. The changes in the image caused by the movements of the camera present another problem, because when 
the camera is moving the entire image changes as far as the video codec is concerned, whereupon even effective 
image coding is unable to maintain satisfactory image quality at the relatively low communk:atkxi speeds available in 
wireless communication. 

Solutrans have been looked tor conceming the problems presented above. One of them is to also display, on the 
25 display of a portable videophone, the image taken by the local camera. However, this solution is poorly adapted for 
use in connect on with portable tenninals, because the size of the display is limited and, in addltkjn, as regards the 
processing power required of the terminal, it is not profitable to display, on one's own display, an image to be transmitted 
(as image data). The additk>nal image restricts the display area available for the display of a received image arKi it 
increases the power consumption of the terminal. This is extremely inconvenient typically in connection with battery- 
30 operated terminals, the operating time of which is reduced. 

In addrtkjn, a solution suitable for use in connection with fixed vkieo conference equipment is presented, as known 
prior art, wherein a camera is focused using a transmitter to be attached on the object to be recorded, which is followed 
by a vkJeo camera installed on top of a video monitor or a projection monitor, typrcally large in size. This solution 
enables, e.g., the automatic monitoring of a teacher, moving in front of the class, in connection with tele-education to 
35 be implemented by means of video conference equipment. The equipment is supplied, at least, by Parker Visbn, an 
Amerk:an company whose product has the registered name of CameraMan. The implementatton of the equipment has 
been described in more detail, e.g., in Patent Publication US 5,179.421. 

SHORT SUMMARY OF THE INVENTION 

40 

A method and a terminal have now been invented for producing an image to be transmitted as image data, by 
means of whbh it is ensured that the user of the terminal can be seen in the image represented by the image data to 
be transmitted by selecting the required section of the image fiekJ recorded by a camera or by focusing the camera. 
The inventktfi will enhance the transmissk>n of a correctly aligned image even when the camera has not been properly 
45 focused or the position between the terminal and its user changes, e.g., when the user's hands move, tt is easy to 
notice, in connectkxi with the presented ennbodiments, that several problems relating to known prior art can be miti- 
gated. 

In the method, according to the inventk>n, the user of the terminal is recorded or photographed (i.e. a representatkxi 
of the user is recorded) for producing an image, to be transmitted as image data, in the terminal and an image, to be 

50 further transmitted as inrtage data, is produced from the videoed or photographed image fiekl. The inventbn utilises 
pattern recognitkxi. The user's figure is kJentified in the recorded image field by comparing the contents of the recorded 
image fiekJ with the figure pre-stored in a memory of the terminal or it is possible to identify the figure, e.g., by means 
of a thermograph. After kientifying the figure (e.g., a figure formed by the user's head and shoulders, a full-length 
portrait or some other figure that needs to be klentified), the locatkxi of the figure in the recorded image field is deter- 

55 mined. When the location of the figure in the image field is known, the focusing of the image, for example, on the user's 
face can be implemented in a number of ways. 

If a video or still camera, equipped with a wide-angle lens, is used in connectbn with the terminal according to the 
present inventkxi, it is possible to determine, e.g., the locatkxi of the user's head and shouWers in the wkJe sector, 
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recorded by the camera, by means of pattern recognition. On the basis of this information, the required section can be 
selected from the recorded image field by means of digital signal processing, e.g., an image field that covers just the 
head and shoulders. If necessary, the image field can also be adjusted whereupon, e.g., only the user's face or the 
whole-length portrait of the user can be displayed of the image field. The image field can be adjusted by the user of 

s the terminal according to the invention or. alternatively, by the opposite end in a remote-controlled manner through a 
separate data channel commonly included in a telecommunication connection. Instead of the user of the terminal, it is 
also possible to identify other things by means of pattern recognition, e.g., figures of different types of objects that you 
want the camera to foltow. Thus, it is possible to apply the invention, e.g., to renrwte controllirig, whereupon it is possible 
to search in the image, e.g., for figures of cars or people and to focus the image, to be transmitted as image data from 

10 the terminal, closer on them. 

If no wide-angle lens is available, it is possible to focus the camera of the terminal at an object, determined by 
means of pattern recognition, by tuming the camera mechanically, e.g., by means of small electric nriotors. If the portable 
terminal is extremely small in size, the use of electric motors is not necessarily the best solution. In this case, it is 
possible to leave the camera for the user of the terminal to focus, and guide the user to turn the terminal in the right 

15 direction, e.g., with the help of signal lights, such as four arrow-like LEDs, to be installed in the terminal. Alternatively, 
it is possible to guide the user by means of characters to be written out on the display (e g., a liquid crystal display) of 
the terminal. 

The invention is characterised in what has been presented in the characterising parts of claims 1 and 4. 

20 PRESENTATION OF FIGURES USED IN THE SPECIFICATION 

Figures 1 A and 1 B illustrate the selection of an image field, to be transmitted, on the basis of pattern recognition 

in an embodiment of a terminal according to the invention, 

25 Figure 2 illustrates the focusing of a camera on the basis of figure identification in another embod- 

iment of a terminal according to the invention. 

Figure 3 illustrates an embodiment of a terminal according to the invention, 

30 Figure 4 illustrates another embodiment of a tenninal, according to the invention, while being used, 

Figure 5 illustrates a third embodiment of a terminal according to the invention, and 

Figures 6A, 6B and 6C illustrate structural parts of the different embodiments of a terminal, according to the inven- 
35 tion, in the form of block diagrams. 

DETAILED DESCRIPTION OF THE INVENTION 

Next, the inventkxi will be discussed in detail using certain embodiments of the terminal according to the invention 

40 as examples, wherein the image to be transmitted as irmge data is selected from the image field recorded by a wkJe- 
angle camera or the camera of a portable terminal is focused on the face and upper body of the user of the terminal. 
In these exemplary embodiments, partem recognition is used for focusing. 

Figure 1 A illustrates a situatbn where a camera 11 , installed in a terminal according to the invention, in this case, 
a portable mobile station 10, takes a picture of a user 5 of the terminal. In this embodiment of the present inventkxi, 

45 the image fieW (Reference P) is selected from the wkie image field (Reference a), recorded by the camera 11 , which 
comprises the head and the upper part of the shoulders of the user (5). The camera 11 used is equipped with a wide- 
angle lens (not illustrated in the figure), so that the user 5 fits in the camera's image field even if the terminal 10 were 
out of line in relatk)n to the user 5. In Figure 18, a wkJe image fieb 17 has been illustrated by dotted lines 12A, 12B, 
14A and 14B. If a focused image fieW 16 was not selected from the wide image fiekJ 17, recorded by the camera 11, 

50 using the method according to the inventbn. a very small upper body image 1 9 of the user 5 would be shown at the 
bcmom of the large image field 17. on the left-hand side. 

The terminal 10, according to the inventk>n, is equipped with pattern recognition means (Reference 61 , Figure 6A), 
which have been arranged to search for a figure 18, having a pre<Jetermined shape (marked in grey in Figure 1 B), in 
the large image field 17, recorded by the camera 11 , in a manner known as such. When the area in the large image 

55 field 17 that sufficiently corresponds with the head and upper body of the user 5 is kx:ated, the tenminal better adjusts 
the focusing of the camera 11 on the face of the user 5. In this particular embodiment, this is implemented so that the 
terminal 10, according to the invention, is equipped with a sufficiently large memory space 64 (Figure 6A), wherein the 
entire large image field 17, recorded by the camera 11, is stored. After this, a required sub-section 62 (corresponds 
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with the image fieJd 16 in Figure 1 B) of the section of the large image field 17, where pattern recognition means 61 
located the area In the image that corresponds with the pre-determined figure 18. is determined in the meniory space 
64, controlled by a controller 63. This new section 1 6 of the image field 1 7, from wh ich the video signal to be transmitted 
is produced, has been presented in Figure IB by dotted lines 13A and 13B, 15A and 15B. The video signal to be 

s transmitted is produced, controlled by the controller 63, by a video converter 65 which retrieves from the memory space 
64, on the basis of the instructions it has received from the controller, the video data contained by the required sub- 
section 62, converts it into a form suitable for a transmitter 66 (e.g., into a digital format according to standard H.261 , 
H.263 or MPEG4 known to a person skilled in the art) and further transmits it to the transmitter 66. This embodiment 
of the terminal, according to the invention, is the portable mobile statbn 10, so the transmitter 66 contains the parts of 

10 the transmission branch of a mobile statbn, known to a person skilled in the art, for transmitting a coded video signal 
through a nrvobile communication network to a receiving terminal (not illustrated in the figure). The receiving terminal 
can be a corresponding type of portable nrxjbile statbn 10. but it can just as well be a terminal, e.g.. video conference 
equipment in contact with a wired telephone network. 

Figure 2 illustrates the focusing of a camera, on the basis of pattern recognrtbn, in another embodiment of a 

15 terminal 25 according to the invention. Figure 6B illustrates functional blocks of this embodiment. Those functional 

blocks that have the same function as in the embodiment illustrated in Figure 1, have been referred to by the same 

numbers. The difference is mainly a camera focusing block 68, controlled by the controller 63, necessary for focusing 
the camera. In this embodiment, the terminal 25, according to the invention, is equipped with a camera 26, having a 
nonmal lens, and an image fieki 20, recorded by the camera, is transmitted to a receiving terminal in whole. In this 

20 respect, this solutbn is simpler and cheaper to implement than the embodiment presented in connection with Figures 
1A, 1B and 6A. In this embodiment, pattern recognitkxi is implemented according to the same principle (Figure 6B, 
Bkx:k 61). The terminal 25 searches, in the image fiekJ 20, for the area in the image that corresponds with a pre- 
determined figure 22, i.e., a figure 21 of the user 5 of the terminal. When it is kxated, the terminal determines the 
difference of the kx:ated area from the centre of the image fiekJ 20. If the figure 21 of the user 5 is located at the bottom 

^5 of the image, on the left-hand side, as is shown in the example illustrated in Figure 2, the terminal 25 adjusts the camera 
26. installed therein, in the required direction so that the figure 21 of the user 5 settles in the centre of the image fiekJ 
20 which, in this exemplary case, means in the directran of an arrow 27. To accomplish this, the terminal 25 is equipped, 
e.g., with snnall electric nrKrtorsand control means (Reference 68, Figure 6B), such as servos, known to a person skilled 
in the art, that adjust the focusing of the camera 26. A sen^o, controlled by the controller 63 of the terminal 25, tums 

30 the camera 26 in the directbn of an arrow 28, whereupon the figure 21 of the user 5 rrxwes ck>ser to the centre. The 
focusing of the camera 26 in the vertical and horizontal direction is implemented according to the same principle. When 
the camera has been focused in both a vertical and horizontal directbn, the figure 21 of the user 5 settles in the centre 
of the image fiekJ and the image is as pleasant as possible from the receiver's viewpoint. 

Figure 3 illustrates another exemplary embodiment of the terminal according to the invention. Contrary to the 

35 embodiment illustrated in Figure 2, it is equipped with a ca mera 32 fi xed in relation to a terminal 30. Thus, it is not 
possible to focus the camera automatically in the manner presented in connectbn with Figure 2 but, instead, this 
structural solution is suited for applications such as the embodiment illustrated in Figu res 1A and IB. In connection 
with the terminal 30, illustrated in Figure 3, the user 5 of the terminal assists in the focusing of the camera. This follows 
that the system operation is not fully automatic but, correspondingly, the structure of the embodiment 30, illustrated in 

40 Figure 3, is very compact and economcal to implement. Hence, it is extremely suitable especially for i>ortable mobil e 
statioris . suitable for video image trans missbn, that operate through a nrK>bile communicatbn network, i.e., for portable 
image phones small in size. For example, the pattern recognition method presented above is used for focusing the 
camera. Because the camera directing means, presented in connectk>n with Figure 2, are unavailable, the information 
on the camera's incorrect position is transmitted to the us er of the termina l 30, who himself focuss the terminal 30 in 

45 the con^ect direction by tu rning the device withjiis hands. The indication of the directkwi in which the camera shoukJ 
be turned is transmitted, e.g., by m earjs of indcato rs. such as arro w-shaped t-EDs 33A. 3 3B. 33C and 33D, illustrated 
I in Figure 3, to be installed on the margins of a displav31 of the te rminal 30. Alternatively, it is also possible to place 
I the indic ators on the dis pla y of the terminal 30 itself, e.g.. on a liquid crystal display, as indicated bv Reference 34. In 
' the smallest and simplest terminals, according to the present invent bn, the indicators mentioned above can be replaced 

50 by a single indicator whk:h indicates whether or not the camera is in the correct positk>n. The controller 63 serves to 
control the LEDS 33A, 33B, 33C and 33D or the LCD 67 on the basis of the results it has received from the pattem 
recognition bkxk 61 . Figure 6C illustrates the structure of the terminal 30 by means of functional blocks. 

Methods for producing an image to be transmitted as image data, according to the inventbn. were presented in 
connectkxi with Figures 1A, IB, 2 and 3, the methods being based on the pattem recognition of the image stored in 

55 the memory 64 as image data. Another way to implement pattem recognitbn is to use, e.g., a thermograph. In this 
case, it is also possible to focus the camera both automatically, as in the embodiments of the inventbn presented in 
connectkjn with Figures 1A, IB and 2, and assisted by the user of the terminal, as in the example presented in con- 
nection with Figure 3. 
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Figure 4 illustrates a tenminal 40, according to the invention, wherein a camera 41 has been focused using a 
thermograph 43. When the terminal 40 is switched on both the video camera 41 and the themrx)graph 43 are activated. 
A temperature detection logic implemented in the tenminal 40, known as such to a person skilled in the art, searches 
for the warmest sectbn in the image fiekJ of the thermograph which, in rTK>st of the terminaFs nonmal operating condh 
s tions. is kxaXed by the face of the user. This has been illustrated by dotted lines 44A and 44B. In more advanced 
systems, it is possible to klentify, in the image fiekJ of the thermograph 43, an isothermal sectkjn in the shape of a face. 
TTiis arrangement prevents interference caused by individual heat sources, kx:ated in the background of the image, 
and also provides for the system to operate in conditbns where the temperature of the operating environment is close 
to the temperature of the face. When the area searched for has been located, an image field 42A and 42B, recorded 

10 by the video camera 41 , is focused on the area around the face to the required tolerance. If the thermograph 43 does 
not kx^ate the place of the face reliably, the video camera 41 is adjusted to the centre positbn to ensure bask: operation. 
If the terminal 40 is not equipped with the camera focusing methods illustrated in Figure 1 A. 1 B or 2, the user can be 
guided to turn the terminal and, at the same time, the fixed camera with the help of the indicatkxi characters 33A-33D 
or 34 as presented in connection with Figure 3. Thus, a thermograph can be used In connectbn with the embodiments 

IS presented above In Figures 1 A, 1 B, 2 and 3. 

The use of the inventbn was described atxjve In connectbn with the portable terminals 10, 25, 30 and 40; for - 

whbh it is partcularly welt suited, because interference caused in the image to be transmitted as image data by the 
movement of the hands and the head can be essentially mitigated by aligning the camera towards the face or upper 
body of the user of the terminal. The Image quality also Improves owing to the fact that as the focusing of the image 

20 remains better unchanged, as regards an image codec, the number of changes taking place In the image decreases 
considerably. Instead of the improvement of the Image quality achieved. It is possible to transmit by means of the 
method for producing an image to be transmitted, according to the Inventkxi, good quality images at a fower transmis- 
sion speed. However, there Is nothing to prevent the inventbn from also being applied in connection with fixed terminals 
designed for office or home use. Figure 5 illustrates a terminal 50, based on computer hardware, which comprises a 

25 so-called multimedia monitor 51 equipped with a camera 52. All the methods, according to the Inventkxi, presented 
above In connection with Figures 1 A, 1 B, 2, 3 and 4, can be used to process the Image data of the camera 52 or to 
focus the camera 52. 

The Implementatkxi and emtxxiiments of the present inventkxi have been presented with the help of examples. It Is 
obvious to a person skilled in the art that the present inventkxi is not restrk:ted to details of the embodiments presented 
30 above, and that the Invention can also be implemented in another form without deviating from the characteristk:s of 
the Invention. The embodiments presented above should be considered Illustrative, but not restrcting. Thus, the pos- 
sibilities of implementing and using the Inventkxi are only restrcted by the enclosed claims. Consequently, the varkxis 
options of Implementing the invention as determined by the claims, including the equivalent implementatkxis, also 
belong to the scope of the inventkxi. 

35 

Claims 

1. A method for producing, in a tenminal (10, 25. 30, 40, 50), an Image to be transmitted as image data, wherein 

40 

an image field (17. 20), comprising a representatkxi of a user (5) of the terminal (10, 25, 30, 40, 50), is recorded 
and an Image (16, 20), Intended for further transmission as Image data, is produced from the recorded image 
field (17, 20) as a response to recording, 

4S characterised In that 

- a figure (1 9, 21 ) of the user (5) is Identified in the recorded image field (1 7, 20). 

- the location of the figure (1 9, 21 ) in the image fieb (1 7, 20) is determined on the basis of the identified figure 
(19, 21) of the user (5), and 

so - on the basis of the determined location, adjustment data (33A, 33B, 33C, 33D, 34) is produced for marking 

off the figure (19, 21) of the user (5) into the image (16, 20) to be transmitted as image data. 

2. A method according to claim 1 , characterised in that 

ss - the recorded image flekJ (17, 20) is stored in an image memory (64). 

- on the basis of the produced adjustment data (33A, 33B, 33C. 33D, 34), a section (62) including the figure 
(19, 21) of the user (5) of the terminal (10, 25. 30. 40, 50) is marked off the image memory (64). 

data included in the section (62) of the marked off image memory (64) is transferred to a vkJeo converter (65) 



5 



EP 0 884 905 A2 



for prcxlucing the image (16, 20) to be transmitted as image data, 

3. A method according to claim 1 . characterised in that 

5 - said adjustment data (33A. 33B. 33C, 33D, 34) is transmitted to the user (5) of the terminal (10, 25, 30. 40, 

50) by means of indication characters (33A, 33B, 33C, 33D, 34) for controlling the directing of the camera and 
for indicating the correct position. 

4. A terminal (10. 25, 30, 40, 50), which comprises 

10 

- a camera (11 , 26, 32, 41 , 50) for recording an image field (1 7, 20) comprising a representation of the user (5) 
of the terminal (10, 25, 30, 40, 50) and for forming image data representing the recorded image field, 

- coding means (62, 63, 64, 65) for coding the image data received from the camera (11 , 26, 32, 41 , 50), and 

- transmission means (66) for transmitting (66) the coded image data to a receiver, 

75 

characterised in that . ... 

- in addition, it comprises pattern recognition means (18, 43, 61, 62. 63, 64) for identifying the figure (19, 21) 
of the user (5) of the terminal (10. 25, 30, 40, 50) and for determining the location in the recorded image field 

20 (17, 20), and 

- adjustment data production means (18, 61 , 62, 63, 64) for producing adjustment data (33A. 33B, 33C, 33D, 
34) for marking off the figure (1 9, 21) of the user (5) into the image (16, 20) represented by the image data to 
be transmitted on the basis of the determined location. 

25 5. A terminal (10, 25, 30, 40, 50) according to claim 4, characterised in that it comprises 

- the image menxtry (64) for storing the image data included in the recorded image field (1 7, 20), 

image mark-off means (18, 61 , 63) for marking off the section (62) of the image memory (64) containing the 
figure (1 9. 21 ) of the user (5) of the terminal (10, 25, 30, 40. 50) on the basis of the produced adjustment data 
30 (33A. 33B, 33C, 33D, 34). 

means (63) for transferring the data, included in the marked off section (62) of the image memory (64), to the 
video converter (65) in order to produce the image (16, 20) to be transmitted as image data. 

A terminal (10, 25. 30, 40, 50) according to claim 4, characterised in that it comprises 

- means (33A. 33B, 33C, 33D, 34) for transmitting the adjustment data (33A, 33B, 33C, 33D, 34) to the user 
(5) of the terminal (10, 25, 30, 40, 50) for focusing the terminal (10. 25, 30, 40, 50). 

7. A terminal (10. 25, 30. 40, 50) according to claim 4, characterised in that said pattern recognition means (18. 43. 
40 61 , 62, 63, 64) comprises a themnograph (43). 

8. A terminal (10, 25, 30, 40, 50) according to claim 4, characterised in that it comprises camera focusing means 
(68) for focusing the camera at the user (5) of the terminal (10, 25, 30. 40, 50) on the basis said adjustment data 
(33A, 33B, 33C, 33D, 34). 

45 

9. A terminal (10, 25. 30, 40. 50) according to claim 4, characterised in that it comprises a mobile station (10, 25, 
30, 40. 50). intended to operate in a mobile communication network, for transmitting image data representing an 
image through the mobile communication network. 
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Fig. 5 
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Fig. 3 



EP 0 884 905 A2 




EP 0 884 905 A2 



11 



10 

5_ 



CAMERA 



61 

PATTERN 
RECOGNmOINl 



IMAGE MEMOR 



62 



64 



18 

4- 



FIGURE 

SEARCHED FOR 
T" ' 



CONTROLLER 



I 



63 



VIDEO 



CONVERTER ^ 



65 



^ V/N TRANSMITTER 



Fig. 6A 



26 



z 



25 

5_ 



61 



18 

4- 



CAMERA 
^ 



z 



^ PATTERN 
RECOGNITION 



IMAGE 
MEMORY 



rv 



64 



FIGURE 

SEARCHED FOR 
T ' 



FOCUSING 
OF CAMERA 



68 




COhiTROLLER 



63 



VIDEO 

CONVERTER 



66 



65 



TRANSMITTER 



lT 



Fig. 6B 



11 



EP 0 884 905 A2 



1 



11 



10 



CAMERA 



61 

PATTERN 
RECXJGNmON 
% 



IMAGE MEMORY 



64 



18 

4- 



RGURE 

SEARCHED FOR 



63 



1 



CONTROLLER 



I 



VIDEO 



66 ^ 



1 CONVERTER Y\j 

I 



_ 33A 
^^33C 



l> LCD 



TRANSMITTER 



67 



Fig. 6C 



12 



